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MULTI-STAGE WATERMARKING PROCESS AND SYSTEM 



Background of the Invention 

Field of the Invention 

5 The present invention is directed to the field of digital rights management. More 

specifically, the present invention is directed to digital watermarking processes and 
systems. 

Description of the Related Art 

In the field of documents and publishing, a work created by an author is usually 
10 provided to a publisher which formats, duplicates and/or prints numerous copies of the 
document. A "document 1 ', as the term is used herein, is any unit of information subject to 
distribution or transfer, including but not limited to correspondence, books, magazines, 
journals, newspapers, other papers, software, photographs and other images, audio and 
video clips, and other multimedia presentations or products. A document may be 
15 embodied in printed form on paper, as digital data on a storage medium, or in any other 
known manner on a variety of media. The copies of the document are then sent by a 
distributor to bookstores or other retail outlets, from which the copies are purchased by end 
users. With the advent of current computing and communications systems such as 
personal computers, workstations, and other devices connected over communications 
20 networks such as local area networks (LANs), intranets, and the Internet, publishers and 
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authors are increasingly publishing their documents electronically to minimize cost and 
delays. Presently, audio and video recordings, software, books, journals and multimedia 
works, are all being electronically published. 

A fundamental issue facing the publishing and other information industries as they 
5 consider electronic publishing, is how to prevent the unauthorized and unaccounted 
distribution or usage of electronically published documents. Electronically published 
documents are typically distributed in digital form and recreated on a computer based 
system having the capability to recreate the published document. The ease in which 
electronically published documents can be accurately reproduced and distributed is a major 

10 concern. Because publishers, authors, as well as other parties in these industries receive 
royalties for each published document that is delivered (i.e. sold), any unaccounted 
distribution of the document such as by copying or distribution to another digital medium 
results in unpaid royalties. Moreover, whereas the low quality of copying and the high 
cost of distributing printed material have served as deterrents to unauthorized copying in 

15 the past, it is far too easy to copy, modify, and redistribute unprotected digital documents. 
Accordingly, some method of protecting digital documents is necessary to make it more 
difficult to copy such documents without authorization. Efforts to resolve this problem 
have been termed "Intellectual Property Rights Management" ("IPRM"), "Digital Property 
Rights Management" ("DPRM"), "Intellectual Property Management" ("IPM"), "Rights 

20 Management" ("RM"), and "Electronic Copyright Management" ("ECM"), collectively 
referred to as "Digital rights management (DRM)" herein. There are a number of issues in 
digital rights management: authentication, authorization, accounting, payment and 
financial clearing, rights specification, rights verification, rights enforcement, and 
document protection, for example. 

25 Two basic schemes have been employed to attempt to solve the document 

protection problem. The first approach is the use of subscription based services in which 
the user is granted access to the document (or information to generate the document) only 
after paying a subscription fee. Existing digital rights management systems such as IBM's 
Cryptolopes utilizes encryption technology to encrypt documents which are then decrypted 
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by users who have paid for the documents or have been otherwise authorized to have 
access to the documents. However, once these documents are distributed and have been 
decrypted by an end user system, little protection is available to prevent copying or 
distributing the decrypted document. In addition, there is no effective way to trace back 
5 such copies of the document to the original source and the intended user. The second 
approach is to utilize proprietary formats wherein the document can only be generated by a 
select application that is obligated to enforce the publisher's rights. Of course, this 
approach requires the use of a single proprietary format and thus, one loses the ability to 
combine popular formats and the richness of content associated therewith. In addition, this 
10 approach requires the user to use a previously unknown application and requires 
development of the select application in order for each format to be created in a secure 
manner. Furthermore, the documents must be generated or converted using non-standard 
tools. 

To deter "pirating", i.e. unauthorized copying, use, or distribution of the 
15 documents, "watermarks" are frequently used to identify the copyright owners, authorized 
users, and/or distributors. Such watermarking also aids in the identification of the source 
of the document thereby allowing enforcement of the rights of the publisher and/or the 
author. With respect to the digitally published documents, current digital watermarking 
technology digitally marks the document with details such as information provided by the 
20 copyright owner, the authorized user, or the distributor. Typically, a watermark is used to 
identify the source or originator of a document. If the watermark is used to identify the 
user to whom the document is delivered, it is typically referred to as a "fingerprint". The 
technologies utilized for both watermarking, and fingerprinting, are typically the same. 
Thus, the term "watermarking" is used herein to refer to both watermarking and 
25 fingerprinting, and the terms "watermark" and "mark" are also used herein to refer to both 
watermarks and fingerprints. 

Presently, most existing watermarking products or tool kits generate marked 
documents via document editors or special tools. For instance, Adobe Photo Shop® is a 
widely used software tool for image creation and publishing on the Internet. DigiMarc 
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Corporation has provided a plug-in to Photo Shop 4.0® which allows a user to embed into 
an image, a digital ID (which is issued by DigiMarc) as well as some attributes such as the 
type of use and content. IBM® has also introduced digital watermarking technology 
through a plug-in for Photo Shop® to identify and preserve the rights of digital content 
5 owners. In addition, watermarking of audio clips such as a music file has also been 
attained by converting the music file into a frequency domain utilizing Fourier transforms, 
embedding a watermarking signal at a desired frequency range, and reconverting the music 
file. In this regard, efforts have been taken to standardize DRM of digital music as 
evidenced by the Secure Digital Music Initiative which may be found in the Internet 

10 address www.sdmi.org. 

Despite the utility that the above-described plug-ins and watermarking techniques 
provide, these watermarking products have several key limitations. More specifically, the 
present applicants have found that the current processes and systems "bind" or affix the 
content information into the watermark at early stages of the document life cycle thereby 

15 limiting the type of content information which can be stored in the watermark to "static" 
information which is known or already determined. Another limitation arise from the fact 
that watermarks are frequently placed into documents manually through the use of 
WYSIWYG (what you see is what you get) editors or other tools. Tools exist for merging 
watermarks into multiple same objects (such as images) of a target document 

20 automatically. However, such tools do not facilitate insertion of various watermarks into 
target documents that have multiple differing objects. Moreover, the use of existing 
processes and systems for inserting watermarks are limited to a single watermarking 
technology and thus, are limited to utilizing one predetermined application. Consequently, 
the existing systems and methods do not allow selection of different or multiple 

25 technologies nor support use of such different technologies. 

Therefore, there exists an unfulfilled need for a watermarking process and system 
which will provide a solution to the above noted limitations to thereby allow efficient 
creation and use of watermarks to identify and preserve the rights of digital content 
owners. 
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Brief Summary of the Invention 

In view of the foregoing, it is an object of the present invention to provide an 
improved process and system that will allow efficient creation and use of watermarks to 
identify and preserve the rights of digital content owners. 
5 A second object of the present invention is to provide an improved process and 

system that will allow early and late binding of content information into the watermark. 

A third object of the present invention is to provide an improved process and 
system that will support the use of multiple watermarking and merging technologies. 

Another object of the present invention is to provide an improved process and 
10 system that will allow automatic or manual merging of watermarks into the document. 

A fifth object of the present invention is to provide an improved process and 
system that will create a watermark specification and render the watermark at different 
stages of the watermarking process. 

Still another object of the present invention is to provide such an improved process 
15 and system that generates a template specification and merges the watermark into the 
document at different stages of the watermarking process. 

Yet another object of the present invention is to provide such an improved process 
and system that allows generation and merging of the watermark into the document at 
different stages of the watermarking process. 
20 These objects are obtained by a multi-stage watermarking system and process in 

accordance with embodiments of the present invention which perform the steps of creating 
a watermark specification which describes how to generate a watermark, generating a 
template specification which describes how to merge the watermark into a target 
document, generating the watermark based on the watermark specification, and merging 
25 the watermark into the target document based on the template specification to provide a 
watermarked document. 

The watermark specification created may include content information which 
describes content to be in the watermark and the watermark may have a plurality of content 
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information associated therewith. The content of each of the content information may be 
application specific and specify at least one of static information and dynamic information. 
The watermark specification created may also include mark technology specification 
which describes a specific watermarking technology to be used in the step of generating 
5 the watermark. The mark technology specification may include parameters needed by the 
specific watermarking technology during the step of generating the watermark. The 
watermark specification created may also include a target object specification which 
describes at least one of a target object and a characteristic of the target object into which 
the watermark is generated, the target object specification specifying at least one of a 

10 shape, color, font and resolution of an image. The target object specification may include a 
pointer pointing to a content information which describes the content to be inserted in the 
watermark. In this regard, the watermark specification created may described using a 
watermark specification language that has grammar elements that: identifies the watermark 
specification, identifies a specific watermarking technology to be used, and identifies a 

15 target object in which the watermark is generated. Optionally, the watermark specification 
language may have grammar elements that at least one of: identifies version of the 
watermark specification language, identifies a name of the watermark specification, and 
specifies technology of content information of the watermark specification. 

In accordance with one embodiment of the present invention, the template 

20 specification generated may include a merge map associated with the watermark 
specification which describes where the watermark is to be merged into the target 
document. A plurality of merge maps may be associated with the watermark specification. 
The template specification generated may include a merge technology specification which 
describes a specific merging technology to be used in the step of merging the watermark 

25 into the target document. The template specification generated may also include merge 
target specification which describes at least one of the target document and a characteristic 
of the target document into which the watermark is merged. In addition, the template 
specification generated may be described using a watermark specification language having 
grammar elements that: identifies version of the watermark specification language, 
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provides a list of a plurality of watermark specifications to be used, and provides a mapped 
list of how a plurality of watermarks associated with the plurality of watermarks is to be 
merged with the target document. In this regard, the watermark specification language 
may optionally have grammar elements that at least one of: identifies the template 
5 specification, identifies a name of the template specification, provides a description of the 
template specification, identifies owner authorized to change the template specification, 
and provides comments regarding the template specification. 

In accordance with another embodiment of the present invention, the step of 
generating, the watermark in the multi-stage watermarking system and process may include 

10 the step of binding dynamic information in the watermark. In addition, the step of 
generating the watermark may also include the step of receiving and interpreting 
information from the watermark specification. The step of generating the watermark may 
also include the step of parsing the watermark specification to thereby obtain the 
information required to generate the watermark. The step of generating the watermark 

1 5 may further include the step of encoding the content that is to be in the watermark. In this 
regard, the specific watermarking technology may be used to place the encoded content in 
a target object to be merged into the target document. 

In accordance with yet another embodiment of the present invention, the step of 
merging the watermark may include the step of receiving and interpreting information 

20 from the template specification. The step of merging the watermark may also include the 
step of parsing the template specification to thereby obtain the information required to 
merge the watermark into the target document. In addition, the step of merging the 
watermark may also include the step of merging the watermarks into the target document 
to thereby provide the marked document. Moreover, , the step of merging the watermark 

25 may also include the step of merging the generated watermark in a target object which is 
then merged into the target document to thereby provide the watermarked document. 

Optionally, in accordance with another embodiment, the multi-stage watermarking 
system and process of the present invention may also include the step of recovering the 
content information from the watermarked document. The content information may be 
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recovered based on at least one of the watermark specification and the template 
specification. 

The above objects are also attained by a system for specifying, generating, and 
merging digital watermarks into a document at different stages of the document's life cycle 
5 in accordance with the present invention including a content information means for 
describing application-specific content to be in watermarks, the application-specific 
content being generated by multiple applications and including application identification 
that is used during watermarks recovery; a static information means for specifying static 
content that is already determined for binding into the watermarks; and a dynamic 

10 information means for specifying dynamic content that is to be determined for binding into 
the watermarks by the time of at least one of generating the watermarked, and creating the 
watermarked document. 

In accordance with another embodiment, the system may also include a mark 
technology specification means for describing a specific watermarking technology to be 

15 used in creating the watermarks, where the mark technology specification means further 
includes parameters needed by the specific watermarking technology. The system may 
also include a target object specification means for providing the target object information 
that describes the target object into which the watermarks are generated. 

The system may also include a target object specification means for providing the 

20 target object information that describes the target object into which the watermarks are 
generated. The data capacity of the watermarks may be determined based on at least one 
of the content information and the target object information. If the data capacity of the 
watermark is insufficient, then the content may be truncated or a pointer to the content 
information may be generated. 

25 The system preferably bounds the static content to the watermarks prior to the 

binding of dynamic content to the watermarks and the watermark is merged into the 
document by at least one of manually by a user and automatically by the system. The 
system may further include a mark specification interpreter which supports multiple 
marking technologies. 
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The above noted objects may be further attained by a system for specifying, 
generating, and merging digital watermarks into a documents at different stages of the 
document's life cycle in accordance with the present invention including a content 
information means for generating application-specific content information to be in the 
5 watermark, the application-specific content information being generated by multiple 
applications and including application identification that is used during watermark 
recovery; a watermark specification means for generating a watermark specification having 
information regarding the watermark based on the content information; a template 
specification means for generating a template specification having at least one merge map 
10 that describes how each watermark is to be merged into a target document; and a 
watermark generation means for generating the watermark in a target object based on at 
least one of a watermark specification, a target object information, and dynamic content 
information. 

In accordance with one embodiment, the content information may also include at 
15 least one of digital rights information, text content information, and reference to a 
workflow processing application. The watermark specification means may include a 
watermark specification interpreter with a parser that parses the watermark specification, 
the watermark specification interpreter adapted to obtain the application-specific content 
information from the watermark specification and to invoke appropriate encoders to 
20 encode the content information that is to be in the watermark. In this regard, the 
watermark specification interpreter may support multiple marking technologies. 

The system may further include a static information means for specifying static 
content that is already determined for binding into the watermarks and a dynamic 
information means for specifying dynamic content that is to be determined for binding into 
25 the watermarks by the time of at least one of generating the watermark, and creating the 
watermarked document. The static content is preferably bound to the watermarks prior to 
the binding of dynamic content to the watermarks. 

In accordance with yet another embodiment, the system may further include a mark 
technology specification means for describing a specific watermarking technology to be 
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used in creating the watermarks. The mark technology specification means may include 
parameters needed by the specific watermarking technology. The system may also include 
a target object specification means for providing the target object information that 
describes a target object into which the watermarks are to be generated. The data capacity 
5 of the watermarks are preferably determined based on at least one of the content and the 
target object information. 

In accordance with still another embodiment of the present invention, the template 
specification may also include a merge technology specification that describes the specific 
merging technology to be use in merging the watermarks into the target document. The 
10 template specification may include a merge target specification that describes the target 
document into which the watermarks are to be merged. 

Preferably, the watermark is merged into the document by at least one of manually 
by a user and automatically by the system. In addition, the template specification is 
preferably generated based on the watermark specification and may include a template 
1 5 specification interpreter that utilizes a parser to parse the template specification. 

These and other objects, features and advantages of the present invention will 
become more apparent from the following detailed description of the preferred 
embodiments of the present invention when viewed in conjunction with the accompanying 
drawings. 

20 Brief Description of the Drawings 

Figure 1 is a schematic illustration of a multi-stage watermarking process in 
accordance with one embodiment of the present invention. 

Figure 2 is a schematic illustration of a watermark specification creation stage in 
accordance with one embodiment of the present invention. 
25 Figure 3 is a schematic illustration of a template specification generation stage in 

accordance with one embodiment of the present invention. 
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Figure 4A is a schematic illustration of a watermark generation stage in accordance 
with one embodiment of the present invention. 

Figure 4B is an detailed schematic illustration of a watermark generation stage of 
Figure 4A. 

5 Figure 5A is a schematic illustration of a watermark merging stage in accordance 

with one embodiment of the present invention. 

Figure 5B is an detailed schematic illustration of a watermark merging stage of 
Figure 5A. 

Figure 6 is a schematic illustration of a watermark recovery stage in accordance 
10 with one embodiment of the present invention. 

Figure 7 is an object model illustration of a multi-stage watermarking process in 
accordance with one embodiment of the present invention. 

Figure 8 is an object model illustration of a content information class in accordance 
with one embodiment of the present invention. 
15 Figure 9 is an object model illustration of a mark technology spec class in 

accordance with one embodiment of the present invention. 

Figure 10 is an object model illustration of a mark technology class in accordance 
with one embodiment of the present invention. 

Figure 11 is an object model illustration of a merge technology spec class in 
20 accordance with one embodiment of the present invention. 

Figure 12 is an object model illustration of a merge technology class in accordance 
with one embodiment of the present invention. 

Figure 13 is one example of a document with watermarks that were generated using 
the multi-state watermarking process of the present invention. 
25 Figure 14 is a schematic illustration of one application where the present invention 

may be effectively used. 

Figure 15 is a schematic illustration of another application where the present 
invention may be effectively used. 
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Detailed Description of the Invention 

As will be evident by the discussions provided hereinbelow, the preferred 
embodiment of the multi-stage watermarking process and system in accordance with the 
present invention allows efficient creation and use of watermarks to identify and preserve 
5 the rights of digital content owners. Initially, it is noted that U.S. patent application Serial 
No. 08/948,893 which is incorporated herein by reference, discloses the use of watermarks 
in identifying and preserving the rights of digital content owners. Moreover, as previously 
noted, it should be kept in mind that the term "watermarking" as used herein below refers 
to both watermarking and fingerprinting, and that the terms "watermark" and "mark" also 

10 used herein refer to both watermarks and fingerprints. 

Figure 1 shows a schematic illustration of a multi-stage watermarking process 2 in 
accordance with one embodiment of the present invention which will attain the above 
noted objectives. As can be seen, the multi-stage watermarking process 2 in the illustrated 
embodiment includes various steps or "stages" including a watermark specification 

15 creation 10 stage, a template specification generation 20 stage, a watermark generation 30 
stage, and watermark merging 40 stage. Moreover, the illustrated embodiment also 
includes an optional watermark recovery 50 stage. These stages of the multi-stage 
watermarking process 2 may be executed by a system such as a computer system (not 
shown) or the like to thereby provide a process and system which avoids the limitations of 

20 the prior art processes and systems. In this regard, it should be initially clarified that the 
present invention may be practiced using a computer system (including distributed and/or 
networked systems) and may be embodied as a software program to be executed in such 
systems. In such an embodiment, the various stages may be embodied or otherwise 
executed as modules of the software program. These modules may be separate from one 

25 another or be integrated together, as long as the modules perform the various desired steps 
or functions as described in the claims appended hereto. 

The multi-stage water marking process 2 and a system of the present invention 
allows the creation of a watermark specification which describes the watermark, and the 
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actual rendering or generation of the watermark, to occur at different stages of the 
watermarking process. For instance, the present invention allows the specification of the 
watermark to be created in a first system, whereas the generation of the watermark occurs 
subsequently on a second system, so long as the second system has access to the 
5 watermark specification created by the first. In addition, the multi-stage watermarking 
process 2 of the present invention also allows the generation of a template specification 
which describes how to merge the watermark into a target document, and the actual 
merging of the watermark into the document, to occur in different stages within the 
watermarking process. Moreover, the multi-stage watermarking process 2 further allows 

10 the generation of the watermark and the merging of the watermark into the document to 
occur at different stages in the watermarking process. As will be explained in further detail 
below, this enables mixing of watermarking technologies within a given document, support 
for various merging technologies, and also allows these stages to occur at different times 
and on different systems. 

15 Importantly, as will be evident to one of ordinary skill in the art in view of the 

discussions below, these provisions of the multi-stage watermarking process 2 allow early 
and late binding of content information of the watermark. In particular, the process in 
accordance with the present invention allows one to have both early binding of static 
information, and late binding of dynamic information in the watermark. Thus, the present 

20 invention allows late binding of dynamic information so that it is possible to incorporate 
content information into the watermarks which are only available or determined at later 
stages of the document life cycle. For instance, the name of the system or device that 
creates the watermarked document or the time at which the watermarked document is 
created can only be determined at the time of their occurrence. Furthermore, the present 

25 invention allows automatic (as well as manual) merging of watermarks into a document. 
In this regard, the present invention allows marking of multiple documents with a single 
watermark specification, as well as marking a single document with multiple watermark 
specifications. Such a feature would be very useful to those who desire to mark a large 
collection of documents using the same set of watermarks. For instance, one can create a 
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template specification for merging multiple watermarks (as defined by watermark 
specifications) into multiple target documents. Moreover, as will be explained in further 
detail below, by providing a multi-stage watermarking process 2, the present invention 
allows multiple watermarking and merging technologies to be used within a document. 
5 The discussion hereinbelow describes in further detail the embodiment of the multi- 

stage watermarking process 2 in accordance with the present invention as shown in Figure 
1. However, for clarity and to facilitate understanding, the discussion hereinbelow sets 
forth the various stages of the watermarking process 2 of Figure 1 noted above as well as 
other aspects of the present invention under separate sub-headings. 

10 Watermark Specification Creation 

As can be seen, one stage of the multi-stage watermarking process 2 in accordance 
with the illustrated embodiment of the present invention is the watermark specification 
creation 10 stage. Figure 2 illustrates the watermark specification creation 10 stage in 
accordance with one embodiment of the present invention in further detail. As can be 

15 seen, the main output of this stage is a watermark specification 12 that describes how the 
watermark is to be generated, what information is to be in the watermark, and what 
technology is to be used in generating the watermark. As will be described in further detail 
below, the watermark may be generated and directly merged into the target document or 
may optionally generated within a specified target object which is then merged into the 

20 target document. In applications where the document is textual, the target object may be 
an area or region in which the watermark is to be placed. In applications where the 
document is an audio clip, a video clip, or other multimedia product, the target object may 
be a video segment or a frequency range or other portion of the multimedia product. It 
should also be noted that various different computer languages can be used to create the 

25 watermark specification 12. However, a preferred embodiment of a watermark 
specification language is described below. As can be seen, the watermark specification 
creation 10 stage utilizes various components to create the watermark specification 12. 
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These various components include content information 14, mark technology specification 
16, and target object specification 18. 

The content information 14 describes the application-specific content that is to be 
inserted into the watermark. It is noted that the content is application-specific which 
5 means that multiple different applications may be used to provide multiple, different 
content information 14 as shown in Figure 2. Correspondingly, it should be appreciated 
that one can have multiple content information 14 (i.e. from multiple applications) within a 
single watermark as long as there is enough data capacity in the watermark. Each of the 
content information 14 also includes the application identification that will be used later in 

10 the multi-stage watermarking process as will be discussed below. In addition, the content 
information 14 may include a pointer which points to the target object where the 
watermark is generated if applicable. The content information 14 may further specify the 
"static" information and the "dynamic" information. In particular, the static information is 
the data/information relative to the watermark or the document that is already determined 

15 or presently known such as the publisher's identity and initial publication date, etc. On the 
other hand, the dynamic information is the data/information relative to the watermark or 
the document that is not determined or presently known but is to be resolved at a later 
time, such as when the watermarked document is created. Such dynamic information may 
include the identity of the authorized distributor or the intended end user who has proper 

20 access to the digital document, which in digital publishing, may not be readily known until 
the document is received, generated and/or printed by the distributor or the intended end 
user. 

The mark technology specification 16 which is also a component of the watermark 
specification 12 describes a specific watermarking technology to be used in generating the 
25 watermark. For instance, the mark technology specification 16 may specify either 
DataGlyph® or DigiMarc®, for example, both of these watermarking technologies being 
commercially available. Other technologies include Verance®'s Electronic DNA®, Alpha 
Tech®'s EIKONAmark®, Audiomark® and Videomark®, Blue Spike®, and IBM® 
applications, etc. The mark technology specification 16 may also include various 
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parameters needed by the specific watermarking technology, such as specific parameters 
specifically required by DataGlyph® or DigiMarc®. 

The target object specification 18 which is an optional component of the watermark 
specification 12 describes the target object into which the watermark may be generated in 
5 one embodiment of the present invention. As previously noted, the target object refers to 
an area, region or a portion in which the watermark may be generated. For instance, the 
target object specification 18 for a DataGlyph® watermark may specify at least one of a 
shape, color, font and resolution of an image into which a DataGlyph® watermark is to be 
generated, the target area then being merged into the target document as described below. 

10 For watermarking technologies that embed watermarks into images, the target object 
specification 18 may specify that the target object is an image. Again, as previously noted, 
in applications where the document is an audio clip, a video clip, or other multimedia 
product, the target object may be a video segment or a frequency range or other portion of 
the multimedia product. It should further be noted that the target object specification 18 

15 may describe a specific target object, or a characteristic of the target object, in which case 
the generation of the watermark may optionally occur at a later stage described 
hereinbelow. It is again noted that the generation of the watermark in a target object and 
merging of the target object into the target document is optional and in other embodiments, 
the watermark may be directly merged into the target document. 

20 In the above described manner, the watermark specification creation 10 stage of the 

present process utilizes content information 14, mark technology specification 16, and 
optionally, target object specification 18 to thereby create the watermark specification 12. 
Thus, the watermark specification 12 describes how the watermark is to be generated, what 
information is to be in the watermark, and what technology is to be used in generating the 

25 watermark. It is also noted that as part of the process of generating the watermark 
specification 12 (or as part of watermark generation described hereinbelow), the size of the 
content information 14 is preferably checked against the data capacity of the watermark 
itself. The capacity of the watermark may be determined based on the mark technology 
specification 16 and the target object specification 18, if applicable. If there is insufficient 
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data capacity in the watermark, various methods may be utilized to address the 
insufficiency. For example, the watermark may be truncated, or a pointer may be provided 
in the watermark to point to the content information 14 rather than putting the actual 
content in the watermark. 

5 Template Specification Generation 

The multi-stage watermarking process 2 in accordance with preferred embodiment 
of the present invention as shown in Figure 1 also includes a template specification 
generation 20 stage which is shown in further detail in Figure 3. The primary output of the 
template specification generation 20 stage is template specification 22 which describes 

1 0 how to merge one or more watermarks directly into the target document, or by merging a 
target object with the watermark as described previously. For instance, the template 
specification 22 may indicate which pages a specific watermark should appear if the 
document is a textual product such as a correspondence or a book. In applications where 
the document is an audio clip, a video clip or some other multimedia product, the template 

15 specification 22 may indicate the audio/visual segments into which the watermarks are 
merged. The template specification 22 may also include a pointer that points to the target 
document. In a similar manner to the watermark specification 12 described above, 
different languages can be developed and/or used for generating the template specification 
22. However, as noted previously, an example of a watermark specification language is 

20 described in a later section below. As can be clearly seen in Figure 3, the template 
specification 22 is generated during the template specification generation 20 stage utilizing 
merge maps 24 that correspond to each watermark specification 12 as described below. In 
addition, optional components such as a merge technology specification 26, and a merge 
target specification 28 may also be utilized in generating the template specification 22. 

25 More specifically, the merge maps 24 describe where each watermark or watermark 

in a target object, as represented by watermark specification 12 of Figure 2, is to be merged 
into the target document. Preferably, there is one watermark specification 12 associated 
with each merge map 24 since each watermark should be merged into the target document 
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in at least one location. Otherwise, there is no need for a watermark specification for a 
watermark which is not used. However, it should be noted that the watermark 
specification 12 may be associated with multiple merge maps 24 since the same watermark 
may be merged into the target document at various locations. 
5 As noted above, the merge technology specification 26 shown in Figure 3 is an 

optional component which may be used to generate the template specification 22 during 
the template specification generation 20 stage. The merge technology specification 26 
describes the specific merging technology to be used in merging the watermarks into the 
target document. For example, the merge technology specification 26 may indicate 

10 PostScript® forms, Document Object Model, XML, MS Office®, etc. which are all 
different technologies. As can be appreciated by one of ordinary skill in the art, this 
description of the specific merging technology is desirable if the merge technology to be 
used requires specific parameters to be defined. 

As previously noted, the merge target specification 28 shown in Figure 3 is also an 

1 5 optional element which may be used to generate the template specification 22 dunng the 
template specification generation 20 stage. The merge target specification 28 describes the 
target document into which the watermarks are to be merged. For instance, the merge 
target specification 28 may indicate that the target document is a correspondence, book, 
magazine, journal, newspaper, other paper, software, photograph, image, audio and video 

20 clips, or a multimedia presentation or product. In this regard, it should be noted that merge 
- target specification 28 may describe a specific target document or a characteristic of the 
target document, in which case binding of the watermark may occur at later stages as 
described hereinbelow. 

Watermark Generation 

25 The multi-stage watermarking process 2 in accordance with one embodiment of the 

present invention as shown in Figure 1 also includes a watermark generation 30 stage as 
shown in Figure 4A. As can be seen, in the illustrated embodiment, the watermark 
generation 30 stage generates a watermark in a target object to thereby provide a marked 
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object 32 as its output based on the watermark specification 12 described previously 
relative to Figure 2. Thus, in this embodiment, the marked object 32 is a watermarked 
target object (i.e. the marked object 32) which may later be merged with the target 
document. Such use of target objects is especially useful in textual documents where 
5 sizing and positioning of the watermark may be required and in HTML applications where 
a specific watermark may only be associated with a specific hyperlink or a specific web 
page. Of course, as previously explained, the present invention may also be practiced 
without the use of such target objects by generating the watermark and directly merging 
the watermark into the target document. However, with respect to the illustrated 

10 embodiment where target objects are used, the various components used in the watermark 
generation 30 stage of the multi-stage watermarking process 2 may be the watermark 
specification 12 (of Figure 2), the target object 34 to which the watermark specification 12 
points to, and dynamic content information 36, if dynamic information is to be bound or 
affixed into the watermark. 

15 The details of one embodiment of the watermark generation 30 stage is more 

clearly shown in Figure 4B. As can be seen, the watermark generation 30 stage includes a 
mark specification interpreter 3 1 which utilizes a watermark specification language (WSL) 
parser 33 to parse the watermark specification 12 to thereby obtain the information 
required to generate the marked object. Such information may include information from 

20 the mark technology specification 16 and/or the mark technology specification 16 
discussed relative to Figure 2. The mark specification interpreter 31 also invokes the 
appropriate content information encoder 35 to encode the content that is to be inserted into 
the watermark. The watermark technology 37 then places the content into the target object 
in the form of watermarks. It should also be noted that in accordance with the embodiment 

25 of the present invention, different watermark technologies 37 can be used to generate the 
watermark. For instance, the watermark technology can be DataGlyph , or any other 
suitable technology. 

By providing a multi-stage water marking process 2 with a watermark specification 
creation 10 stage and a separate watermark generation 30 stage, the present invention 
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allows the creation of a watermark specification 12, and the actual generation or rendering 
of the watermark to occur at different stages of the watermarking process 2. Thus, the 
present invention allows the specification of the watermark to be created in one system, 
whereas the generation or rendering of the watermark may occur subsequently on a 
5 different rendering device. 

Watermark Merging 

The multi-stage watermarking process in accordance with one embodiment of 
present invention also includes watermark merging 40 stage in which the watermark 
generated during the watermark generation 30 stage is merged into the target document. 

10 As previously explained, the watermark may be directly merged into the target document. 
However, if the watermark was generated in a target object to thereby provide a marked 
object 32, the target object with the watermark therein (i.e. the marked object 32) may be 
merged into the target document. As shown in Figure 5A, a watermarked document 42 is 
outputted by merging the watermark or the marked object into the target document 44. It 

15 should be noted that whereas in the present illustrated embodiment, the target document 44 
(and thus the resulting watermarked document 42) is an actual document, in other 
embodiments, the target document 44 may alternatively be another object so that the 
watermarked document 42 is another watermarked object that can later be merged into a 
document. Thus, the term "document" should be construed broadly to mean actual 

20 documents or objects. As can be also seen in Figure 5A, the watermarked document 42 is 
provided in the watermark merging stage 40 based on various components including the 
previously described template specification 22 of Figure 3, and a target document 44 to 
which the template specification 22 points to. 

The specific details of one embodiment of the watermark merging 40 stage is more 

25 clearly illustrated in Figure 5B. As can be seen, the watermark merging 40 stage includes 
a template specification interpreter 41 which utilizes the watermark specification language 
parser 43 to parse the template specification 22 of Figure 3 to thereby obtain the watermark 
specification 12 and the mapping requirements as provided by the merge maps 24. The 
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template specification interpreter 41 then invokes the merge technology 45 to merge the 
watermarks or the target objects with the watermarks therein (i.e. marked objects 32) into 
the target document to thereby provide the watermarked document 42. It should also be 
noted that different merging technologies can be used to generate the watermarked 
5 document 42, depending on the format and the requirements of the watermarked document 
42. For instance, there may be specific merge technologies for PostScript (via forms) or 
for MS Office, etc. 

In the manner described above, the multi-stage watermarking process 2 of the 
present invention thus allows the creation of template specification 22 and the actual 

10 merging of the watermark into the document to occur in different stages within the 
watermarking process 2. Moreover, the multi-stage watermarking process 2 further allows 
the generation and merging of the watermark into the document at different stages in the 
watermarking process. This enables mixing of watermarking technologies within a given 
document and also allows these stages to occur at different times and on different systems. 

15 As can now be appreciated by one of ordinary skill in the art, these provisions of the multi- 
stage watermarking process 2 allows early binding of static information and late binding of 
the dynamic information in the watermark. The present invention also allows automatic 
(as well as manual) watermarking of multiple documents with a single watermark 
specification, as well as watermarking a single document with multiple watermark 

20 specifications. Moreover, the multi-stage watermarking process 2 of the present invention 
provides support for various watermarking and merging technologies. 

Watermark Recovery 

The multi-stage watermarking process 2 in accordance with the embodiment of the 
present invention as illustrated in Figure 1 also includes an optional watermark recovery 50 
25 stage which is more clearly shown in Figure 6. In the watermark recovery 50 stage, the 
content information 14 (Figure 2) which was previously put into the watermark of the 
watermarked document 42 in the manner described above, is now recovered from the 
watermarked document 42. This may attained by utilizing an object character recognition 
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(OCR) system known in the art to thereby obtain desirable information regarding the 
watermarked document. For instance, the recovery of the watermark may be desired to 
recognize the type of document, determine the document's authenticity, the document's 
source, or other relevant information. The template specification 22 and the watermark 
5 specification 12 which were also described previously relative to the other stages of the 
multi-stage watermarking process 2, may also be utilized to effectively recover the content 
information 14. 

Object Model 

To further describe the multi-stage watermarking process 2 in accordance with the 

10 illustrated embodiment of the present invention of Figure 1, the discussion hereinbelow 
sets forth the present invention utilizing an object model syntax. In particular, Figure 7 
describes an embodiment of the present invention utilizing the Unified Modeling Language 
(UML) which is a general purpose notational language used by those skilled in the art to 
specify and visualize complex software. As can be seen in Figure 7, the relevant outputs 

15 of the stages and the classes corresponding to the components thereof are illustrated in 
Figures 1-6 and have been recreated as an object model using the UML syntax. 
Correspondingly, the relevant outputs of the stages and the classes have been enumerated 
in Figure 7 utilizing numerals of Figures 1-6 which have been increased by 100. It should 
also be noted that due to the length of the text in the object models, the term "watermark" 

20 is merely referred to as "mark" which as previously noted, refer to both watermarks and 
fingerprints. In addition, the term "specification" is merely referred to as "spec" whereas 
"technology" is sometimes referred to as "tech". 

Accordingly, in Figure 7, the watermark specification is represented by mark spec 
112 while the template specification is represented as template spec 122. The mark spec 

25 112 and template spec 122 represent the data and the actions that can be performed on the 
previously described watermark specification 12 and template specification 22 
respectively. As can be seen, the mark spec 112 includes the classes content information 
1 14, mark technology spec 116, and target object spec 118. In the illustrated embodiment, 
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the mark spec 1 12 may utilize the mark technology 137 to generate the watermark. As can 
also be seen, the template spec 122 includes classes merge map 124, merge technology 
spec 126, and merge target spec 128. In the illustrated embodiment, the template spec 122 
may utilize the merge technology 145 to properly merge the watermark into the target 
5 document to thereby provide a watermarked document. Thus, Figure 7 clearly shows a 
representative relationship between the mark spec 112, the template spec 122, and the 
various classes thereof. 

As shown in Figure 8, the content information 114 may be subclassed into 
application specific objects. In the illustrated example, the content information 114 may 

10 include rights info 202, the text content info 204, and workflow applet 206 (i.e. a workflow 
processing application). The rights info 202 is content information that may include digital 
property rights information. The text content info 204 is an application that inserts straight 
text information into the watermark. The workflow applet 206 may represent the various 
applets that will be invoked during different stages of the document workflow. It should 

15 again be noted that the content information 114 is a representation of various application- 
specific content information 14 described previously and thus, it may vary depending on 
the specific application. 

The mark technology spec 1 16, as shown in Figure 9, and the mark technology 137, 
as shown in Figure 10, represent various technologies that can be used in generating the 

20 watermark in accordance with the multi-stage watermarking process of the present 
invention. As can be seen in Figure 9, the mark technology spec 116 may include the 
DataGlyph® tech spec 208 or the DigiMarc® tech spec 210 which correspond to Xerox® 
DataGlyph® and DigiMarc® technologies respectively that may be utilized to create the 
watermark specification 12 shown in Figure 2. Similarly, as shown in Figure 10, the mark 

25 technology 137 may utilize the DataGlyph® mark tech 209 or the DigiMarc® tech 211 to 
generate the desired watermarked object 32 shown in Figure 4A and 4B. Thus, with 
respect to Figures 9 and 10, two different watermark technologies are represented under 
the watermark specification creation 10 stage and the watermark generation 30 stage 
described previously of the multi-stage watermarking process in accordance with one 
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embodiment of the present invention. It should also be noted that other watermark 
technologies may also be used in other embodiments of the present invention. As 
previously noted, such other technologies include Verance®'s Electronic DNA®, Alpha 
Tech®'s EIKONAmark®, Audiomark® and Videomark®, Blue Spike®, and IBM® 
5 applications, etc. 

In a similar manner, the merge technology spec 126, and the merge technology 145 
of Figures 11 and 12 represent the various merging technologies that can be used into 
merge the watermark into the target document in accordance with the present invention. In 
particular, as can be seen in Figure 11, the merge technology spec 126 may include 

10 PostScript® PS forms tech spec 212, document object model (DOM) tech spec 214, or MS 
Office® tech spec 216 that may be used in the template specification generation 20 stage to 
generate the template specification 22 as shown in Figure 3. In addition, as shown in 
Figure 12, the merge technology 145 may include the PostScript PS forms merge tech 
213, document object model DOM merge tech 215, or MS Office® merge tech 217 that 

1 5 may be used in the watermark merging 40 stage in the manner shown in Figures 5 A and 
5B to thereby generate the watermarked document 42. Thus, with respect to Figures 11 
and 12, two different merging technologies are represented for the template specification 
generation 20 stage and the watermark merging 40 stage described previously of the multi- 
stage watermarking process in accordance with one embodiment of the present invention. 

20 Of course, it should also be noted that other merging technology may be used instead of 
the ones discussed above. 

It can be readily appreciated by one of ordinary skill in the art, the above described 
multi-stage watermarking process in accordance with the present invention attains the 
previously noted objectives by allowing digital watermarks to be specified, generated, and 

25 merged, into documents at different stages of the document life cycle. Thus, the present 
invention enables mixing of watermarking technologies within a given document, and 
enables support for various merging technologies, and also allows these stages to occur at 
different times and on different systems. The present invention also allows early binding 
of static information and late binding of dynamic information. The present invention 
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further allows automatic watermarking of multiple documents with a single watermark 
specification, as well as watermarking a single document with multiple watermark 
specifications. 

5 Example Applications 

Figure 13 illustrates a watermark template 302 having various watermarks 304 
generated in the manner described above which may be utilized in various applications, 
two of such applications being described in further detail hereinbelow which are made 

10 possible by the multi-stage watermarking process of the present invention. Figure 14 
illustrates one application for the multi-stage watermarking process in accordance with the 
present invention. In this application, a publisher utilizes a template creation system 402 
described in further detail hereinbelow to sell digital documents that are to be printed by 
users at their local commercial print shops that utilize print system 404. To deter and/or 

15 detect illegal copying, the publisher desires to put standard watermarks for all of their 
documents printed at the commercial print shops. The publishing company creates a 
watermark template 302 of Figure 13 or any other watermark template that includes a 
template specification described previously. The watermark template may specify several 
watermarked objects that are to appear on each printed document. For instance, the 

20 publisher may specify borders for its standard header and footer using DataGlyph 
watermarks. Of course, different sets of watermarks could be specified for the front cover 
and/or a dynamic information may be put into the watermarks in order to identify the 
various print shops. These templates can then be distributed to print shops that are 
authorized to print the publisher's documents. When the print shop prints the publisher's 

25 document, the print system 404 will follow the watermark template to render the correct 
watermarks on the specified pages. 

More specifically, as can be seen in Figure 14, the template creation system 402 
used by the publisher includes a mark specification tool 410 and a template tool 420. The 
mark specification tool 410 enables the users to create watermark specification described 
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previously, and stores it into the watermark specification repository 411. The publisher 
interacts with the template tool 420 to thereby create templates having template 
specifications described previously by using the watermark specification stored in the mark 
specification repository 411. The templates are then stored in the publisher template 
5 repository 421. These templates are subsequently stored in the printer template repository 
441 of an authorized print shop. Any method can be used to transfer the templates from 
the publisher's template creation system 402 to the print shop's print system 440 including 
via a network such as the Internet, a secure network, or by physically transferring an 
information storage media such as an optical or magnetic disc with the templates thereon. 

10 The templates stored within the printer template repository 441 of the print system 440 
may then be retrieved as needed by the print system 440. Of course, in other applications, 
the templates may also be provided together with the document to be printed. 

When a publisher's document 460 is submitted to the print shop, the print system 
440 spools the publisher's document 460 into the printer document spool 445. The print 

15 system 440 then searches the printer template repository 441 for the appropriate template 
to use. During rendering, the printer's rendering module 447 performs the watermark 
generation and watermark merging function described previously by generating the 
watermark, reading the template, and merging the watermark into the target document to 
thereby provide a watermarked document 461. The document may then be and printed by 

20 the printer marking module 449 and a hard copy may be outputted by the print system 
440. Any dynamic content info 446 may be provided to the printer rendering module 447 
so that such dynamic content info 446 can be placed into the watermark of the 
watermarked document 461 by the printer marking module 449. As previously explained, 
such dynamic content info 446 may be any information which was not readily available, 

25 for instance, the identity of the print shop or the identity of the person for whom the 
watermarked document 46 1 is provided. 

Figure 15 illustrates yet another application of the multi-stage watermarking 
process in accordance with the present invention. In this illustrated embodiment, the scan 
service 502 utilizes the multi-stage watermarking process to process various forms in the 
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manner described below. The scan service 502 may be, for example, an insurance 
company which processes various types of claim forms or documents that are sent into the 
company via surface mail or fax. The scan service 502 scans in each form or document for 
record keeping purposes and inserts watermarks into the scanned image of each claim form 
5 or document in order to record various types of information such as the receipt date of the 
form and the status of the claim. Other information can of course, be provided in the 
watermark to facilitate workflow processing of the claim form. The scan service 502 may 
design different watermark templates for each type of claim form 560, each template being 
defined using a watermark specification to record the information in the appropriate format 

10 (for example, DataGlyph®), and a template specification in the manner described 
previously. For instance, the template specification may include a watermark to record the 
status of the claim as a text image in the background of the claim form image. This may 
be desirable to allow claims adjusters to process the claim forms manually. 

Referring again to Figure 15, when the claim form 560 is initially scanned in by the 

15 scan service 502, the scan service 502 performs the watermark recovery function 
corresponding to the watermark recovery 50 stage and looks for the form type which has 
been recorded in the DataGlyph format. The image processing system 547 then brings up 
the corresponding watermark template from the template repository 521 and puts the 
desired additional watermarks into the image. Moreover, dynamic content information 546 

20 such as information regarding the current status of the claims may also be incorporated 
into the watermark. The image processing system 547 then performs the watermark 
generation and watermark merge functions described previously to render the watermarks 
into the image and output the watermarked image 562 which may then be sent to an 
appropriate repository for subsequent processing. 

25 

Watermark Specification Language 

As previously noted, the present invention provides a multi-stage watermarking 
process 2 and a system such as a computer based system for executing such a process. In 

NVA 163858.1 




-28- 



DocketNo. 111325-35 



this regard, one embodiment of a watermark specification language (hereinafter "WSL" or 
"language") that may be used for defining the watermark specification 12 and template 
specification 22 discussed above is proposed herein below in detail. This embodiment of 
the watermark specification language is modeled from the Digital Property Rights 
5 Language (DPRL) developed by Mark Stefik et al. as fully described in detail in U.S. 
Patents No. 5,629,980, and No. 5,715,403 which are incorporated herein by reference, as 
well as application Serial No. 08/948,893 which was previously incorporated herein by 
reference. As such, the language grammar and grammar rules given below relative to the 
watermark specification language generally correspond to DPRL but the keywords and the 

10 syntax of the WSL differ from DPRL. It should also be noted that DPRL itself includes 
information which may be a component of the watermark specification. In particular, 
DPRL specifies the requirement for watermark from the rights perspective. For example, 
the publishers can specify what information is to be in the watermark. The WSL in 
accordance with the present invention also allows specification of how the watermark 

1 5 should be put into the document. In this sense, WSL enhances DPRL and may be used in 
conjunction with DPRL to define the watermark specification 12, or be used by itself. This 
is attained since the WSL also allows defining of the template specification 22 in 
accordance with the present invention as described previously. Of course, other languages 
may be utilized in practicing the present invention and the details set forth hereinbelow 

20 should be understood as merely showing one example. For example, the present invention 
may be practiced by utilizing a watermark specification language in the XML format or 
other derivations thereof 

Initially, it is noted that terms of the WSL in italics are expanded into other terms in 
the actual WSL specification, the specification referring to the watermark specification 12 

25 and/or the template specification 22 in the present embodiment. Italicized terms that are to 
be expanded into other terms in the Actual WSL statements are referred to as non- 
terminals, and terms that appear in actual WSL statements are referred to as terminals. The 
right arrow "->" indicates a grammar expansion rule. Thus, as an example, the rule: 
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a-spec -> b: b-spec 

means that the non-terminal a-spec is expanded to "b:" followed by an expansion of b- 
spec. The subscript "opt" indicates that a specification is optional. Vertical bars " I " are 
used to indicate alternative expressions. For example, the rule 

5 b-spec— > a | 

x (Goodies: b-list) opt \ 
c: c-spec 

means that a b-spec is expanded into either an "a" or an "x" followed optionally by a 
parenthetical expression starting with "Goodies:" and followed by the expansion of b-list, 

10 or a "c:" followed by an expansion of c-spec. 

In the embodiment below, the white space in the language such as spaces and tabs, 
is used merely to separate the grammar of the language and has no other meaning except 
inside quoted strings. The case of terms such as capitalization is also not meaningful. 
Thus, a "Template:" keyword can be written as "Template:" or "template:" or even 

15 "tEmPlaTe:". However, capitalizing the first letter is preferred for readability. Preferably, 
programs that print specifications out from objects utilize the preferential capitalization. 
Some of the design goals for the grammar include: 

• Simple parsing. To arrange statements in uniform format to thereby simplify 
parsing. 

20 • Extensive defaulting. To provide a language with well-defined defaults when a 

particular parameter is not specifically given, especially since many parameters 
are optional. 

• Extensiblity. To establish a language style in which it is easy to add new 
features unambiguously without substantially modifying the language or 

25 rewriting old specifications. 

To this end, the grammar elements have the form: 
(keyword: parameteri parameter 2 ) 
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that is, a keyword is followed by a colon, followed by a set of optional parameters. The 
language has many optional parameters and few position-sensitive parameters. The 
parameters themselves are recursively represented as further elements. This leads to 
sentences of the form: 



with short parenthesized elements whose meaning is easy to recognize. At the bottom, 
there are keyword-value pairs that describe particular parameter values. 

In the examples below, the preferred order for key-word phrases is the order shown 
10 in the grammar. In many cases, this order is thought to be clearest in meaning. However, 
except as noted in the text, compilers and interpreters of the language preferably accept the 
phrases in other orders as well. Programs that generate statements in the language are 
expected to print out phrases in the preferred order. 

Template Specification 

15 The highest level grouping in the Watermark Specification Language (WSL) is 

template specification 22 (Figure 3) which as been described previously above and may 
have the following grammar with the grammar elements: 



5 



(keyword] :(keyword2: parameter]) 

(keyword 3: (keyword^ parameter) (keywords: parameters) )) 



20 



25 



watermark-template —> 
(WMTemplate: 
(WSL- Version: version-id) 
(Template-ID: template-id) opt 
(Name: text-description)^ 
(Description: text-description)^ 
(Owner: certificate-spec) op i 
(Comment: comments tr) opt 
watermark-list-spec 
m ap-l ist-spec) 
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The WSL- Version includes a grammar element which is a machine-readable 
identifier version-id that identifies the version of the watermark specification language 
used. The identification of the version of the WSL used facilitates backwards 
compatibility to works in which the template specification were written in an older version 
5 of the language. 

As indicated by the subscript "opt", the grammar element template-id is an optional 
identification tag which can be used to identify the particular template. In this regard, the 
template-id may identify a digital certificate signed by a trusted registration authority that 
assigns a unique identification number to the template. It also may include information for 

10 locally detecting any changes or tampering with the template. In addition, the template-id 
may also include address information for locating a reference copy of the template. 

The template specification 22 may also include optional grammar elements for a 
name and a description of the template. The format and scope of the information in the 
name and the description are not specified so that different parties can put various types of 

1 5 information here, depending on the application. 

The template specification 22 may further include an optional grammar element for 
owner specification, which may indicate who is authorized to change any part of the 
template-specification, including adding or deleting watermarks and its associated 
parameters. The owner specification may also indicate what digital license is required to 

20 make the changes. If no owner specification is specified, then anyone (or no one) would 
be able to make changes to the watermarks in a digital work. 

The template specification 22 may also include an optional grammar element for 
comments. Comments in WSL may be intended for documentation purposes and may be 
used by people creating and updating template specifications. Comments are not 

25 interpreted for meaning by watermarking systems but their provision and use is important 
in the WSL. More specifically, since WSL statements can be compiled into objects and 
subsequently printed out again to make new source statements, it is important that 
comments be preserved throughout the edit-compile-reprint cycle for specifying rights. In 
the WSL herein described, the comments have fixed and specific locations in the syntax 
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("structured comments") corresponding to structured locations for object storage to thereby 
ensure that the comments be stored in the compiled watermark objects. In this regard, 
there is preferably only one comment in a template specification at each level. 

The watermark-list-spec in the template specification 22 preferably lists all of the 
5 watermark specifications that are used in the template and have the following grammar: 

watermark-list-spec -> (Watermark-List: watermark-list) 
watermark-list — > wm-spec watermark-list opt 

A fuller description for the watermark-spec is provided in a later section hereinbelow. 

The map-list-spec in the template specification 22 lists the descriptions of how each 
10 watermark should be merged within the document and have the following grammar: 

map-list-spec -> (Map-List: map-list) 
map-list — > map-spec map-list opi 

A fuller description for the map-spec is also provided in a later section hereinbelow. 

An example template specification 22 is given below: 

15 (WMTemplate: 

(WSL- Version: 1.01) 

(Template-ID: "Joumal-Press-Registry-kjru89902) 
(Name: "standard-journal-page") 

(Description: "Template for page for standard journal cover page") 
20 (Owner: (Certificate: 

(Authority: "Journal Publishers Association") 
(ID: "Journal Press Publishing"))) 
(Comment: "To be included in all journal cover sheet") 
(Watermark-List: 
25 (WMSpec: ( 

(Name: "DataGlyph 1X6") 
(WMSpec-ID: "JPP-1278" 
(WMContentlnfo.RightsContentlnfo: 
(Label-ID: JPA22 3 049-01)) 
30 (WMTechlnfo.XDGTechlnfo: . . .) 
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(WMTargetlnfo :...)) 
(WMSpec: ( 

(Name: "Copyright Notice") 
(WMSpec-ID: "JPP-748922" 
5 (WMContentlnfo.NullContentlnfo: ))) 

) 

(Map-List: 

(WMMap: ( 

(WMSpec-ID: "JPP-1278") 
10 (WMPosition.PageCoord: 1 50 125))) 

(WMMap: ( 

(WMSpec-ID: "JPP-748922") 
(WMPosition.PageCoord: 2, 350 250))) 

) 

15 ) 

Watermark Specification 

Watermark specification 12 (Figure 2) defines how the watermark is to be 
generated, what information is to be in the watermark, and what technology is to be used in 
generating the watermark. It contains information on the content (such as content 
20 information 14), the technology to use (such as mark technology specification 16), and the 
target object (target object specification 18) into which the watermark is to be placed. It 
should be noted that the watermark specification 12 can exist independent of the template 
specification 22 described previously and has the following grammar with the grammar 
elements: 

25 wm-spec -> 

(WMSpec: 

(WSL- Version: version-id) opi 
(Name: name-str) opt 
(WMSpec-ID: wm-spec-id) 
30 content-info-list opt 
tech-info-spec 
target-info-spec 
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The WSL- Version includes a grammar element which is a machine-readable 
identifier version-id that identifies the version of the watermark specification language 
used. The identification of the version of the WSL used facilitates backwards 
compatibility to works in which the watermark specification were written in an older 
5 version of the language. The provision of the version-id is optional for the watermark 
specification 12 if the watermark specification 12 is actually a part of the template 
specification 22, and if the version of the language happens to be the same. Each 
watermark specification 12 may have an optional grammar element for a name: 
Name: name-str 

10 The watermark specification 12 also includes a grammar element wm-spec-id 

which is an identification tag used to identify the watermark specification 12. This is 
preferably provided since each watermark map-spec appearing in the template 
specification 22 refers to a watermark specification 12 using the wm-spec-id. 

As previously noted, there are preferably three main components to the watermark 
15 specification 12: the content information 14, the mark technology specification 16, and the 
target object specification 18 which are shown in Figure 2. As previously described, the 
content information 14 indicates what information should be in the watermark. In many 
cases, the content information 14 is application specific. For example, a rights 
management application may specify rights-related information to be in the watermark 
20 whereas a workflow application may specify other types of information to be in the 
watermark. Therefore, the parameters for the content information 14 is preferably specific 
to the application and in the present embodiment, has the following grammar: 
content-info-list —> content-info-spec content-info-list opt 
content-info-spec -> (WMContentlnfo xontent-info-qualifier: 
2 5 application-id-spec 0 pt 

content-info-parameters) 
application-id-spec -> Appl-ID: application-id 
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The grammar element content-info-qualifier specifies the particular application and thus, 
the content-info-spec, for example, may be: 

(WMContentlnfo.RightsContentlnfo: (Label-ID: 25.457/12546)) 

where the RightsContentlnfo is the application and the Label-ID is the parameter specific 
5 to the application. 

The grammar element application-id is an optional identification tag in the 
watermark to help identify the particular application associated with the content so as to 
aid in associating the application with the content when the watermark is recovered. 

The mark technology specification 16 describes a specific watermarking 
10 technology, with associated parameters, to use in creating or interpreting the watermark. 
Thus, the specification is closely related to the specific technology selected. For example, 
an application involving DataGlyphs® may require error correction parameters, whereas 
one involving DigiMarc® may us some other particular information. Because of this 
specificity, the technology information parameters in the language may depend on the 
1 5 technology actually being used. 

tech-info-spec— > 

(WMTechlnfo.technology-info-qualifier: technology— info-parameters) 

Here the grammar element technology-info-qualifier identifies the specific technology 
involved and the technology-info-parameters provide the necessary parameters for that 
20 technology. For example, the tech-info for DataGlyphs may be: 

(WMTechlnfo.XDGTechlnfo: (Error-correction: 20) (Write-white: false)) 

The target object specification 18 describes the target object into which the 
watermark is to be placed and may have the grammar: 

target-info — > (WMTargetlnfo .target-info-qualifierJarget-info-parameters) 



NVA163858.1 





-36- 



DocketNo. 111325-35 



This specification is loosely related to the watermarking technology and the content 
information. For instance, it isn't meaningful to specify a JPEG image as the target object 
if the technology is specified as DataGlyphs®. In the above noted grammar, the target- 
info-qualifier is specific to the type of the target. For example, if the technology selected 
5 is DataGlyphs®, the target-info may be: 



specification 22 discussed above, the map-spec defines how the watermark is to be merged 
into the document and may have the grammar with the grammar elements: 

map-spec — > 
(WMMap: 



wm-pos-spec) 

The grammar element wm-spec-id is an identification tag which may be used to identify 
the corresponding watermark specification 12 and is assumed to be in the template 
specification itself, or otherwise be available to the application that processes the template 

20 specification. The map-spec and the positional information therein, indicates the specifics 
of where the watermark should appear in the target document. In many cases, the 
positional information is specific to a target object. For example, if the target object is a 
document formatted for printing, the positional information may be in terms of pages and 
the x/y coordinates within a page. Positional information may also point to some object in 

25 the document. If the target object is a video or audio clip, the positional information could 
be quite different (e.g., with time element). In any case, the grammar element wm-pos- 
spec may be: 



(WMTargetlnfo.XDGTargetlnfo: 
(XDGWidth: 16) 
(XDGHeight: 16) 
XDGFileType: BMP)) 



10 



Lastly, referring again to the map-spec previously noted relative to the template 



(WMSpec-ID: wm-spec-id) 
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wm-pos-spec -> (WMPosition.position-qualifier: position-vector) 
position-vector -> position-element position-element-vector opt 

The position specification is preferably in a form of a vector representing the hierarchy of 
positional elements, with each successive element defining the position in more detail. For 
5 example, position specification for a document might have section, chapter, page and x/y 
position in the vector, in that order. The position-qualifier specifies the particular type of 
position specification. For example, the wm-pos-spec may be: 

(WMPosition.PageCoord: 1 546 785) 

The type and the order of individual elements in the position-vector are specific to 

10 the type of the position specification as indicated by the position-qualifier. In the example 
above, the PageCoord qualifier identifies the page number and the x/y coordinates within 
the specified page. There may be other page based qualifiers that describe in different 
forms, the page designation (e.g., first page, last page, etc.). Each position-element itself 
could also be of the "(a: parani)" form, depending on the type of the position element that 

1 5 the position vector requires. 

It should be evident from the above detailed discussions that by providing a multi- 
stage water marking process, the present invention allows the creation of a watermark 
specification, and the actual generation of the watermark, to occur at different stages of the 
watermarking process. It should also be evident how the present invention allows the 

20 generation and merging of the watermark into the document to occur at different stages in 
the watermarking process. It is further evident how the present invention allows the 
creation of template specification and the actual merging of the watermark into the 
document to occur in different stages within the watermarking process. As explained 
above, these provisions allow early binding of static information and late binding of 

25 dynamic information. The present invention further allows automatic watermarking of 
multiple documents with a single watermark specification, as well as watermarking a 
single document with multiple watermark specifications. Moreover, it should also be 
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evident how the present invention allows the support of multiple watermarking and 
merging technologies so that it will allow various watermarking technologies to be used 
within a document. 

It should also be noted that the present invention can be embodied as software, 
5 hardware, or combination thereof. For instance, the present invention may be implemented 
utilizing any type of hardware or combination thereof such as various "clients" and 
"servers". Accordingly, the terms "client" and "server" as used herein, can refer to any 
type of computing device or data terminal, such as personal computer, a portable computer, 
a dumb terminal, a thin client, a hand held device, a wireless phone, or any combination of 

10 such devices. The various clients and servers can be a single computer at a single location 
or multiple computers at single or multiple locations. For example, a server may be 
comprised of a plurality of redundant computers disposed in co-location facilities or at 
various locations to facilitate scalability. There can be any number of clients and any 
number of servers and the client can also be physically located on the same hardware as the 

15 server. 

Moreover, the present invention can be implemented over any type of 
communications network, such as the Internet, a local area network (LAN), a wide area 
network (WAN), direct computer connections, or the like using any type of 
communications hardware and protocol. Communication can be accomplished over 

20 electric cable, fiber optic cable, or any other cable, or in a wireless manner using radio 
frequency, infrared, or other technologies. As previously noted, documents can be of any 
type and contain any type of content such as text, audio information, video information, or 
any combinations thereof. 

It should also be noted that whereas the multi-stage watermarking process in 

25 accordance with the present invention is described as having multiple stages, it should be 
appreciated that these stages need not be embodied in separate software or hardware but 
may be provided in a single software or hardware. Moreover, it should also be apparent 
that the various stages of the present embodiment as discussed above can be subdivided 
into small stages and other stages may be added as well. In this regard, if the present 
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invention is implemented in software, the various steps can be merely separate portions of 
the same software code, different files, interspersed lines of code, etc. 

While various embodiments in accordance with the present invention have been 
shown and described, it is understood that the invention is not limited thereto. The present 
invention may be changed, modified and further applied by those skilled in the art. 
Therefore, this invention is not limited to the detail shown and described previously, but 
also includes all such changes and modifications which are encompassed by the claims. 
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